Paper 15.3 Food and You 2: Declining response rates and the impact on representation
On this page
Skip the menu of subheadings on this page.Summary
Since 2020 we have observed a gradual decline in response rates to our Food and You 2 survey (from 30.4% to 25.6%). This is not unique to Food and You 2 and similar trends have been observed in other government surveys. As we prepare for the next wave of fieldwork (Spring/Summer 2026), we are considering if there are any additional measures we should put in place to mitigate the risk of further reductions to response rates and the potential reduction in the representativeness of the data. Members are asked to consider:
1. How concerned should we be about the current downward trend in Food and You 2 response rates?
2. What additional measures, if any, could we put in place to mitigate the risk of further reductions to response rates and the potential reduction in the representativeness of the data?
3. If this downward trend continues, should we adjust how we present and interpret the survey results?
Background
Food and You 2 is our official statistic survey with consumers. We have been conducting the survey bi-annually since 2020 (from 2025 it will be annual). The survey is conducted among a cross-section of adults (aged 16 years or over) living in households in England, Wales, Northern Ireland and Scotland. Adults invited to take part are selected from a sample of the Royal Mail’s Postcode Address File (PAF) using a random probability sampling methodology. Up to two adults per household are eligible. The survey is conducted using a push-to-web methodology whereby participants are contacted by post and invited to complete an online survey. Those who choose not to complete the online survey, after the initial reminder, are subsequently sent a postal version of the survey.
Strategies to maximise survey response rates
A recent literature review conducted as part of the Economic Social Research Council (ESRC)-funded Survey Futures project examined how various strategies used in self-completion surveys with address-based sampling frames affect response rates.
The review found that the following strategies are effective at increasing response rates:
- Offering different modes to take part (e.g. online and postal)
- Offering multiple methods of accessing the survey e.g. URL, QR code
- Sending pre-notifications
- Sending survey reminders, with 2 or 3 reminders being the optimum number
- Shorter intervals (1.5 - 3 weeks) between reminders
- First class postage and special delivery on outgoing mail
- Stamped return envelopes and multiple stamps on return envelopes
- Use of incentives, with monetary and unconditional incentives being most effective
How we maximise response rates in Food and You 2
We have a number of measures in place to maximise Food and You 2 response rates and ensure the data is representative of the population (see Table 1).
Table 1. Strategies used to maximise response rates in Food and You 2.
| Stage | Strategy | Description |
| Pre-fieldwork / survey design | Sample Size | Ahead of each wave we adjust the issued sample size based on response rates from previous waves. As response rates have declined, we have increased the size of our initial sample. |
| Pre-fieldwork/ survey design | Letter design |
Letters are designed based on the Tailored Design Method, developed by Professor Don Dillman, as well as previous studies conducted on other government surveys1. Letters include the FSA, Defra and Ipsos logo, and are signed by the FSA’s Deputy Director for Analysis. The letters include FAQs to reassure recipients how their data will be stored. Letters are also personalised based on the region of the recipient. |
| Pre-fieldwork/ survey design | Survey length | The online survey length is limited to 30 minutes and the postal survey is no more than 20 pages. |
| Pre-fieldwork/ survey design | Mobile-first | The survey is designed with a ‘mobile-first approach’ meaning that it is designed with smart phone users in mind initially, as this is how most participants respond. |
| Pre-fieldwork/ survey design | Cognitive and user testing | Questions have undergone cognitive testing to ensure questions are clearly understood. User testing was also conducted before Wave 1 to ensure the survey and letters are user-friendly. |
| Pre-fieldwork/ survey design | Accessibility | The survey is designed to be as accessible as possible. For example, the online survey can be adjusted to increase font size and adjust colour contrasts, in Wales the survey and letters are provided in English and Welsh, and respondents are able to request to complete the survey by telephone in another language or receive a large font / Braille copy of the survey |
| Fieldwork | Mixed mode | Respondents can take part in the survey online or via post to encourage those who do not have access to the internet or are not comfortable going online to participate (around 30% of respondents opt to take the postal survey). |
| Fieldwork | QR codes | Letters include a QR code to make it quick and easy to access the online survey. |
| Fieldwork | Envelopes and stamps | During Wave 11 fieldwork, digital printed stamps were trialled to see if this increased response rates compared to a standard franked mail mark. Three mailings were split into two equal groups – one receiving the digital stamp and the other the franked mark. 51% of survey responses were received from addresses that received the digital stamp envelopes, and 49% from those with franked mail envelopes |
| Fieldwork | Incentives | Respondents are offered a £10 voucher (online or paper) for taking part. |
| Fieldwork | Number of mailings and reminders | Households receive up to 4 mailings (approx. 2 weeks apart); an initial invitation letter and 3 reminder letters. |
| Fieldwork | Additional reminders | An additional reminder letter can be issued if fieldwork monitoring indicates we are unlikely to meet our target. |
| Fieldwork | Fieldwork timings | We have the flexibility to extend fieldwork by keeping the online survey open for longer, or allowing more time for respondents to complete and return the postal questionnaire. |
| Fieldwork | Reserve sample | A reserve sample is drawn at the same time as the main sample and can be used to increase the number of respondents if after 4 weeks of fieldwork monitoring indicates we are unlikely to meet our target. |
| Post-fieldwork | Weighting |
Weighting is applied to the data to account for selection and non-response bias, and calibrate the data to the population levels2. |
1More detail on the letter design principles can be found in the technical report (page 21)
2Data is weighted for age band within gender, geographic area, and deprivation quintile. More detail on weighting can be found in the technical report (page 26).
Incentivising early response
In Wave 1 of Food and You 2 we conducted an experiment to see if we could encourage online responses by manipulating the incentive regime. Incentives were varied across three experimental groups: control group respondents received a £10 conditional incentive; experimental group 1 received a £15 conditional incentive if they responded before the first reminder despatch or a £10 one if they responded later; experimental group 2 also received a £15 conditional incentive if they responded before the deadline but only a £5 one if they responded later.
Offering the larger incentive to those responding before first reminder despatch (£15 instead of £10) increased the number of responses received before that deadline. Compared with the control group, experiment group 1 (£15 or £10) retained a higher questionnaire return rate at the end of fieldwork. In contrast, in experiment group 2 (£15 or £5), the early questionnaire return rate gain was completely lost by the end of fieldwork, and the final questionnaire return rate was below that of the control group. Comparisons of selected demographic and survey variables between the three groups provided no evidence that different incentives affected survey estimates. Costs per achieved respondent were around 5% greater for both the early incentive groups relative to the control. Based on these findings, we decided to continue using a £10 conditional incentive for all respondents. The full findings are published in the Social Research Association (SRA) Journal.
Food and You 2 response rates
How have response rates changed over time?
The overall address level response rate varied up and down from 28% to 31% in the first four waves of Food and You 2. But since Wave 5, there has been a clear pattern of decline (see Figure 1). Response has decreased at every wave by an average of 0.7 percentage points. By Wave 10, it was down to 26%.
The same general pattern of decline was repeated in England, Wales and Northern Ireland. The main difference between countries is the lower response rate in Northern Ireland (it has dropped from 26% to 23% over this period).
Figure 1. Address level response rate (%) in Food and You 2 by Wave, England, Wales and N Ireland, Waves 1 to 10
Has mode of response changed over time?
Response rates are often dependent on the mode used to collect the data. However, the ‘push-to-web’ design of Food and You 2 has remained the same over all ten waves. The proportion of online responses has varied from around 60% to 70% with no obvious pattern.
What impact is this having on the sample profile?
Our chief concern with declining response rates is that it can create bias in the survey results. This is not automatic. Bias will only occur when non-respondents differ considerably from respondents in ways that affect the survey’s outcome. For this reason, we want the sample to remain representative despite any non-response. When response rates vary between population sub-groups, this can lead to non-response bias, particularly when the affected groups are related to the survey responses. Reduced response also reduces the sample size. This will reduce the precision of survey estimates unless more resources are put into recruitment.
The decline in responses rates seems to have only had minor impacts on the demographic profile of the sample. After the first four waves, there have only been minimal differences in age, gender and household size. The biggest change has been to ethnicity, with the proportion of Mixed/Asian/Black/Other respondents increasing from 6% to 8%. (For context, 16% of respondents aged 16+ in the 2021 Census were of Mixed, Asian, Black, or of Other ethnicity).
Conversely, the first four waves saw larger changes. The proportion of 16-44 year-olds in the sample decreased from 37% to 33%. This decline was mainly offset by an increase from 28% to 32% in those aged 65+. (Older respondents are over-represented in the Food and You 2 sample. Based on recent ONS population data, 46% of the UK 16+ population are aged 16+ 16-44 but only 23% are aged 65+.). Also, the proportion of respondents in one-person households dropped from 17% to 13% (offset by an increase in two-person households).
What impact is this having on survey costs?
Lower response rates also reduce the sample size, and therefore the precision of estimates. To counteract this, in the last wave we issued a larger sample size to ensure we received the target number of responses. Although this proved successful this costed the FSA extra in terms of printing and postage costs. We are therefore keen to explore strategies that can maintain or improve current response rates without increasing survey costs.
What could be contributing to declining response rates?
The first four waves of Food and You 2 took place during the period of COVID-19 restrictions, which could have influenced response in many ways.
The response rate for Food and You 2 may be declining for similar reasons to other social surveys. Whereas response rates of 70% were common in social surveys thirty years ago, there has been a global decline since then. Now response rates are typically below 50%, with some online surveys having response rates of between 20% to 30%3.
Possible reasons for lower participation in government social surveys might include:
- A decline in trust in government and research institutions
- Increased concerns around privacy and sharing personal and sensitive information
- Lower responses to unsolicited data requests due to an increased awareness of scams
- Survey fatigue caused by frequent requests to complete marketing surveys and service evaluations
- A large increase in the proportion of the population aged over 80 (they might be less able to make use of the internet or are more likely to have health difficulties)
- Inflation reducing the impact of the current £10 completion incentive
3For a examples of how response rates have declined in other government surveys, please see Annex 4 of Systemic Review of ONS Economic Statistics.
Discussion
Members are asked to consider:
- How concerned should we be about the current downward trend in Food and You 2 response rates?
- What additional measures, if any, could we put in place to mitigate the risk of further reductions to response rates and the potential reduction in the representativeness of the data?
- If this downward trend continues, should we adjust how we present and interpret the survey results?